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DETAILED ACTION 
Claim Rejections - 35 USC § 103 

1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1 and 1 1 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Kanade et al. (US Patent 6,084,979 A) in view of Kim et al. (US Patent 6,314,21 1 B1). 

Regarding claim 1 , Kanade et al. discloses a method (fig. 4) for computing a 
fused depth map (fig. 4, num. 30) of a view of an object (fig. 4, num. 2), comprising: 

a) obtaining camera calibration data for a new view of an object ( Using 
figure 1, Kanade et al. states," The information captured by the scene transcription 
hardware 3 is used by an internal representation generator 4 to create an internal 
representation (col. 5, lines 17-20)."). Note that an internal representation is a camera 
angle of the scene at col. 2, lines 60-63, and that internal representations are derived 
from scene descriptions or camera calibration data. Note that scene descriptions are 
camera angles with corresponding pixels that represent depth at col. 4, lines 62-64.; 
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b) rendering each of a plurality of known depth maps of the object into the 
new view of the object based on the camera calibration of the new view (Using figure 2, 
Kanade et al. states," The method of the present invention may be carried a step further 
in which the internal representation 4 is implemented by creating object centered 
models 13 from the depth information. When object centered models are created, in 
effect CAD models of the objects are created. Thus, a view of the object from any 
viewing angle can be generated using standard CAD techniques (col. 5, lines 53-59) ."; 
and 

c) computing the depth map elements(or scene descriptions) of a fused 
depth map (Fig. 4, num. 30:"lmage Fusion") of the new view of the object, wherein each 
said depth map element corresponds to a pixel in the new view of the object (The scene 
description contains pixel and camera angle information for creating a new view) and 
wherein each said depth map element comprises of a set of rendered depth map 
elements, corresponding to the pixel, each from one of the plurality of rendered depth 
maps (Five slightly different depth maps are extracted at fig. 4, num. 26) (Using figure 4, 
Kanade et al. states," The purpose of the step of image fusion 30 is to integrate all the 
scene descriptions into one object-centered 3D model of the scene... (col. 8, lines 40- 
42)." Note that scene descriptions are camera angles with corresponding pixels that 
represent depth at col. 4, lines 62-64). 
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Kanade et al. does not teach the use of a median value for the fusion step, but 
does teach the use of a robust fusion algorithm that corrects depth errors during the 
combination of the scene descriptions at col. 8, lines 43-46. 

However, Kim et al., in the field of endeavor of three-dimensional imaging, does 
teach the use of a median value to generate a composite or fused image. Kim et al. 
states, "...the median disparity is extracted among the aligned blocks and the disparities 
of all blocks [or sets]... are replaced by the median disparity... At this time, the disparity 
map is also modified (col. 16, lines 51-44)." Note that Kim teaches that a disparity 
between two images provides a value of depth at col. 1 , lines 59-64. Thus the disparity 
maps in Kim et al. corresponds to the depth map of Kanade et al. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify Kanade et al.'s robust fusion algorithm that corrects depth errors 
with the teaching of Kim et al.'s median disparity extraction, because the use of Kim et 
al.'s median disparity extraction improves "the quality of the new image... When the 
motion is measured and the new image is generated according to the above-mentioned 
block matching method, blocks may be shown in the generated new image or the 
quality may be deteriorated in the edge area. In order to compensate for this, a 
predetermined post-process, for example, filtering using a median filter is performed 
(Kim et al., col. 12, lines 38-48)." 
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Claim 1 1 is similar to claim 1 , except for the following: 

Regarding claim 1 1 , Kanade et al. teaches a system (fig. 4) for computing a 

fused depth map (fig. 4, num. 30) of a view (fig. 4, num. 2) of an object (fig. 3), 

comprising: 

a measurement device (fig. 4, num. 3 corresponds with fig. 1 , num. 3). 
a merging device (fig. 4, num. 30). 
a processor (fig. 4, num. 28). 

The remaining functions performed by these devices in of claim 1 1 have been 
respectively addressed in claim 1. 

3. Claims 9,10,19 and 20 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kanade et al. (US Patent 6,084,979 A) in view of Kim et al. (US 
Patent 6,314,21 1 B1 ) and further in view of Martens et al. (US Patent 6,252,974 B1 ). 

Claim 19 is similar to claim 1, except for the following: 

Regarding claim 19, Kanade et al. and Kim et al. teaches a system (Kanade et 
al., fig. 4) for constructing a graphical representation of an object (fig. 15), as perceived 
from a desired view (Kanade et al. states, "...a view of the object from any viewing angle 
can be generated using standard CAD techniques (col. 5, lines 57-59."), from a plurality 
of calibrated views (fig. 4, numerals 2), comprising: 

a measurement device (fig. 4, num. 3 corresponds with fig. 1, num. 3) obtaining 
camera calibration data (This element was addressed in claim 1a). 
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a first processor (Kanade et al., fig. 4, numeral 26 is part of a computer or 
processor of figure 4, numerals 26,28,30,32 and 33 and also depicted in fig. 1 , num. 4 
(col. 5, lines 20,21 and col. 10, lines 29-31) that extracts depth information stored in 
fig. 4, num. 3 and also depicted in figure 1, num. 3Thus, Kanade et al.'s 
computer of figure 1, num. 4 is a processor which is also depicted within figure 4, 
numerals 26,30,32, and 33.) computing a plurality of depth maps (Figure 4, num. 26 
uses a MBS technique that extracts dense depth maps of the images as shown in figure 
5a-5e at col. 7, lines 1-5 and 14,15), each depth map (fig. 5a-5e at col. 8, lines 14-19 or 
"another image" as described below.) respectively corresponding (Kanade et al. finds a 
correspondence between a calibrated reference image and another image at col. 7, line 
63,64 and col. 8, lines 17-19) to each of a plurality of calibrated views of the object 
(Kanade et al. uses a calibration object for each camera before the MBS technique at 
col. 7, lines 47-49; on the other hand, uses the MBS technique for generating a depth 
map from a reference image or calibration object from either 5a-5e at col. 8, lines 14-19. 

); 

a merging device (Kanade et al., fig. 4, num. 28 merges data from numerals 
30,32 and 33.) rendering a selected, computed depth map into the desired view for 
each of the plurality of computed depth maps (Kanade et al. states, "To render an 
object, event, or environment from viewing angles other than those where cameras are 
actually located from scene descriptions, a graphic workstation 28 (FIG. 4) may be 
provided to perform the function of the view generator 5 of FIG. 1 (col. 1 1 , line 63 thru 
col. 12, line 1)." Note that scene description created in fig. 4, num. 3 are inputted to fig. 
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4, num. 26 which creates depth maps from figures 5a-5e using the MBS stereo 
matching technique.; and. 

a third processor (The first processor as described above is the same as the 
third processor which comprises a part fig. 4, num. 30 that computes a fused depth 
map.) computing a fused (fig. 4, num. 30) depth map value (Fig. 4, num. 26 creates 
depth maps using the MBS stereo matching technique that is inputted into fig. 4, num. 
30. ) at each pixel (The MBS technique recovers dense depth maps, i.e., a depth 
estimate corresponding to every pixel at col. 7, lines 14-16) in the desired view (The 
desired view is created using the workstation 28 of fig. 4). 

The combination of Kanade et al. and Kim et al. does not teach occlusion as 
required in for claim 19, but Kanade et al. does suggest a method for occlusion at col. 
13, lines 6-22. 

However, Martens et al., in the field of endeavor of depth measurements, teach 
occlusion of four views (fig. 2, num. 205 and 225) wherein as many rendered depth 
maps occlude said pixel as pass it by (Fig. 2, third column, third row shows a case of 
equal visible "1" and non-visible "-1" views due to occlusion. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the teachings of Kanade et al.'s occlusion and fusion with 
Martens et al.'s teaching of occlusion, because Martens et al.'s method of occlusion "is 
the simplest way of estimating depth over several frames... (Martens et al., col. 6, lines 
51,52.) 
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Regarding claim 20, the combination of Kanade et al. teaches the system 
according to claim 19, further including an image projection device (fig. 4, num. 
6:"MONITOR") projecting the graphical representation of the object as perceived from 
the desired view (addressed in claim 1), utilizing the computed fused depth map 
values. 

Kanade et al. from the Kanade combination does not teach a median depth 

value. 

However, Kim et al. from the Kanade combination does teach the use of a 
median disparity map or depth map to generate an image at col. 12, lines 38-48. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the depth maps of Kanade et al. with the median teaching of 
Kim et al. for the same reasons as claim 1 . 

The limitations of claim 9 have been addressed in claim 19 above, and 
thoses of claim 10 have been addressed in claim 20 above. 

Allowable Subject Matter 

4. Claims 3-8,13-18 allowed. 

Claims 2 and 12 objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. 

The following is a statement of reasons for the indication of allowable subject 
matter: 
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The combination of the prior art does not suggest finding a depth area using 
occlusion of pixels with a support matrix for determining the depth area in combination 
with all the other features in claims 2,3,7,13, and 17. 

Conclusion 

5. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Minamida et al. (US Patent 6,633,664 B1) is pertinent as teaching a method of 
using camera calibration data to create a three-dimensional structure at fig. 2. 

Jasinschi et al. (US Patent 6,504,569 B1) is pertinent as teaching a method of 
obtaining a median of an depth map based on camera parameters at col. 10, lines 61- 
64. 

Snyder (US Patent 6,266,064 B1 ) is pertinent as teaching a method of selecting 
pixel based on occlusion (fig. 5). 

Tirumalai et al. (IEEE) is pertinent as teaching a method of using a median with 
fusion of disparity maps at page 467, left column. 

6. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dennis Rosario-Vasquez whose telephone number is 
703-305-5431. The examiner can normally be reached on 9-5. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo Boudreau can be reached on 703-305-4706. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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